Abstract:

Cognitive diagnosis has recently gained prominence in educational assessment, psychiatric evaluation, and many other fields. Researchers have been trying their best to develop a new Cognitive Diagnosis Model (CDM) or to improve existing ones’ performance for respondent classification. As a new CDM, GDD (Sun, Zhang, Xin, & Bao, 2011) receives more and more attention due to its classification accuracy which is as high as DINA. This article introduces a new approach called Hamming Distance Discrimination (HDD) which is inspired by GDD and based on the Q-matrix theory (Tatsuoka, 1991) modified by Leighton et al. (2004) and Ding et al. (2009, 2010). HDD uses Hamming Distance (HD) to measure the distance between an examinee’s Observed Response Pattern (ORP) and an Expected Response Pattern (ERP). When there are more than one ERPs with the same minimum HD for an examinee’s ORP, two solutions based on HD are proposed: the random method (Method R) and the Bayesian method (Method B). Method R randomly chooses one ERP from those share the same minimum HD whereas in method B, we apply Bayesian Discriminant to distinguish which ERP the examinee belongs to. Monte Carlo simulation was used to compare the accuracy of respondent classification between HDD and GDD. In the Monte Carlo simulation study, the pattern match ratio and average attribute match ratio were used as criteria to evaluate the classification accuracy of GDD and HDD. Five attribute hierarchical structures in attribute hierarchical model (AHM) of Leighton et al. (2004) and Tatsuoka (1995, 2009) with 6 attributes were simulated. Under each type of Q-matrix, we set the slip at four levels (2%, 5%, 10%, 15%) to simulate ORPs of examinees (N=1000). The results of this study demonstrate that HDD is superior, especially under the unstructured hierarchy and independent structure. Moreover, method B presented higher classification accuracy than method R. Further research on HDD’s validity and performance in other situations is warranted.

Key words: GDD, Q-matrix, knowledge states, Hamming Distance